THE MODALITY OF TEXT IN MULTIMEDIA INSTRUCTIONS refining the design guidelines

According to cognitive load theory (Sweller, 1999) and Mayer's theory of multimedia learning (Mayer, 2001), replacing visual text with spoken text (the modality effect) and adding visual cues relating elements of a picture to the text (the cueing effect) increase the effectiveness of multimedia instructions. The aim of this study was to test the generalisability of both effects in a classroom setting. The participants were 111 second-year students of educational science (age between 19 and 25 years), who studied a web-based multimedia lesson on instructional design for about one hour and completed a retention and a transfer test. During instructions and tests, self-report measures of mental effort were administered. Adding visual cues to the pictures resulted in higher retention scores, whereas replacing visual text with spoken text resulted in lower retention and transfer scores. A possible explanation for the reversed modality effect is that the instructions were learner-paced, and not system-paced like in earlier research. The use of multimedia computers in education has led to the development of all sorts of instructional material in which verbal and non-verbal presentation modes are combined. Unfortunately, educational research has not yet identified how to design effective multimedia instructions. However, two recent lines of research that have yielded some promising results are the work on cognitive load theory by Sweller and others (for an overview, see Sweller, 1999) and the experiments on multimedia learning carried out by Mayer and colleagues (for an overview, see Mayer, 2001). Both researchers base their instructional design principles on human cognitive architecture and the way in which the multimedia material is processed. In his theory of multimedia learning, Mayer (2001) describes how the learner builds mental representations of multimedia instructions. One important step in this process is the integration of both verbal and visual information in working memory. For example, when instructions consist of a picture and an explanatory text, the learner has to switch back and forth between the two to integrate them mentally. This process is cognitively demanding, at the expense of mental resources that could otherwise be allocated to the learning process. * based on: Tabbers, H. K., Martens, R. L., & Van Merriënboer, J. J. G. (in press). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology. EFFECTS OF MODALITY AND CUEING 20 Cognitive load theory calls the unnecessary memory load caused by the presentation format of instructions extraneous load (Sweller, Van Merriënboer & Paas, 1998). Changing the presentation format can lower this extraneous load and increase the effectiveness of instructions. For example, Sweller and others have shown that the physical integration of verbal and visual information resulted in improved test scores (Chandler & Sweller, 1991, 1992; Kalyuga, Chandler & Sweller, 1998; Sweller, Chandler, Tierney & Cooper, 1990; Tarmizi & Sweller, 1988). When a textbox is placed right next to the part of the picture the text is referring to, the need to mentally integrate text and picture is reduced, which lowers the extraneous load and facilitates the learning process. Sweller et al. (1998) call this the split-attention effect. A similar effect has been demonstrated by Mayer and colleagues in a series of experiments in which they showed that multimedia instructions were more effective when verbal and visual information were presented close to each other rather than spatially separated (Mayer, 1989; Mayer, Steinhoff, Bower & Mars, 1995; Moreno & Mayer, 1999). Mayer (2001) calls it the contiguity principle. A more recent finding is that multimedia instructions can be more effective when the verbal information is presented auditorily instead of visually. This is called the modality effect (Sweller et al., 1998) or modality principle (Mayer, 2001). A number of experiments have demonstrated that replacing written or on-screen text with spoken text improved the learning process in different ways: lower mental effort during instruction and higher test scores (Tindall-Ford, Chandler & Sweller, 1997), less time on subsequent problem solving (Jeung, Chandler & Sweller, 1997; Mousavi, Low & Sweller, 1995), and improved scores on retention, transfer and matching tests (Kalyuga, Chandler & Sweller, 1999, 2000; Mayer & Moreno, 1998; Moreno & Mayer, 1999). The authors explain these diverse results by referring to the working memory model of Baddeley (1992). In this model, working memory has two modality-specific slave systems, one for visual and spatial information and one for acoustic information. When information is presented in two sensory modalities (visual and auditory) rather than one, both slave systems are addressed and total working memory capacity is used more efficiently. So, relative to the available resources, the extraneous load of the multimedia instructions is reduced. Both strategies, physically integrating text and picture and replacing written or on-screen text with spoken text, reduce the extraneous load of multimedia instructions and thus increase the effectiveness of the learning process. In both cases, this reduction in cognitive load can partly be accounted for by the reduction in the amount of visual search needed to integrate text and picture. The effect of reducing visual search has been explicitly demonstrated in two studies by Jeung et al. (1997) and by Kalyuga et al. (1999). Jeung et al. showed that replacing visual text with spoken text does not always improve the effectiveness of multimedia instructions, especially when pictures with a high visual complexity are used. Only when they added to the pictures in the bimodal condition visual cues that related the right elements in the picture to the accompanying spoken text, did they recover the modality effect in terms of shorter time on subsequent problem solving. Kalyuga et al. found the same cueing effect with visual-only instructions. In one experiment they used colour-